MCMC based Generative Adversarial Networks for Handwritten Numeral Augmentation

This is the author accepted manuscript. The final version is available from Springer via the DOI in this record.In this paper, we propose a novel data augmentation framework for handwritten numerals by incorporating the probabilistic learning and the generative adversarial learning. First, we simply transform numeral images from spatial space into vector space. The Gaussian based Markov probabilistic model is then developed for simulating synthetic numeral vectors given limited handwritten samples. Next, the simulated data are used to pre-train the generative adversarial networks (GANs), which initializes their parameters to fit the general distribution of numeral features. Finally, we adopt the real handwritten numerals to fine-tune the GANs, which increases the authenticity of generated numeral samples. In this case, the outputs of the GANs can be employed to augment original numeral datasets for training the follow-up inference models. Considering that all simulation and augmentation are operated in 1-D vector space, the proposed augmentation framework is more computationally efficient than those based on 2-D images. Extensive experimental results demonstrate that our proposed augmentation framework achieves improved recognition accuracy.This work was supported by grants from the Chinese Scholarship Council (CSC) program

Designing fuzzy rule based classifier using self-organizing feature map for analysis of multispectral satellite images

We propose a novel scheme for designing fuzzy rule based classifier. An SOFM based method is used for generating a set of prototypes which is used to generate a set of fuzzy rules. Each rule represents a region in the feature space that we call the context of the rule. The rules are tuned with respect to their context. We justified that the reasoning scheme may be different in different context leading to context sensitive inferencing. To realize context sensitive inferencing we used a softmin operator with a tunable parameter. The proposed scheme is tested on several multispectral satellite image data sets and the performance is found to be much better than the results reported in the literature.Comment: 23 pages, 7 figure

Collaborative Layer-wise Discriminative Learning in Deep Neural Networks

Intermediate features at different layers of a deep neural network are known to be discriminative for visual patterns of different complexities. However, most existing works ignore such cross-layer heterogeneities when classifying samples of different complexities. For example, if a training sample has already been correctly classified at a specific layer with high confidence, we argue that it is unnecessary to enforce rest layers to classify this sample correctly and a better strategy is to encourage those layers to focus on other samples. In this paper, we propose a layer-wise discriminative learning method to enhance the discriminative capability of a deep network by allowing its layers to work collaboratively for classification. Towards this target, we introduce multiple classifiers on top of multiple layers. Each classifier not only tries to correctly classify the features from its input layer, but also coordinates with other classifiers to jointly maximize the final classification performance. Guided by the other companion classifiers, each classifier learns to concentrate on certain training examples and boosts the overall performance. Allowing for end-to-end training, our method can be conveniently embedded into state-of-the-art deep networks. Experiments with multiple popular deep networks, including Network in Network, GoogLeNet and VGGNet, on scale-various object classification benchmarks, including CIFAR100, MNIST and ImageNet, and scene classification benchmarks, including MIT67, SUN397 and Places205, demonstrate the effectiveness of our method. In addition, we also analyze the relationship between the proposed method and classical conditional random fields models.Comment: To appear in ECCV 2016. Maybe subject to minor changes before camera-ready versio

An integrated approach to prognosis using protein microarrays and nonparametric methods

Over the past several years, multivariate approaches have been developed that address the problem of disease diagnosis. Here, we report an integrated approach to the problem of prognosis that uses protein microarrays to measure a focused set of molecular markers and non-parametric methods to reveal non-linear relationships among these markers, clinical variables, and patient outcome. As proof-of-concept, we applied our approach to the prediction of early mortality in patients initiating kidney dialysis. We found that molecular markers are not uniformly prognostic, but instead vary in their value depending on a combination of clinical variables. This may explain why reports in this area aiming to identify prognostic markers, without taking into account clinical variables, are either conflicting or show that markers have marginal prognostic value. Just as treatments are now being tailored to specific subsets of patients, our results show that prognosis can also benefit from a ‘personalized' approach

A comparison of machine learning methods for classification using simulation with multiple real data examples from mental health studies

Background: Recent literature on the comparison of machine learning methods has raised questions about the neutrality, unbiasedness and utility of many comparative studies. Reporting of results on favourable datasets and sampling error in the estimated performance measures based on single samples are thought to be the major sources of bias in such comparisons. Better performance in one or a few instances does not necessarily imply so on an average or on a population level and simulation studies may be a better alternative for objectively comparing the performances of machine learning algorithms. Methods: We compare the classification performance of a number of important and widely used machine learning algorithms, namely the Random Forests (RF), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA) and k-Nearest Neighbour (kNN). Using massively parallel processing on high-performance supercomputers, we compare the generalisation errors at various combinations of levels of several factors: number of features, training sample size, biological variation, experimental variation, effect size, replication and correlation between features. Results: For smaller number of correlated features, number of features not exceeding approximately half the sample size, LDA was found to be the method of choice in terms of average generalisation errors as well as stability (precision) of error estimates. SVM (with RBF kernel) outperforms LDA as well as RF and kNN by a clear margin as the feature set gets larger provided the sample size is not too small (at least 20). The performance of kNN also improves as the number of features grows and outplays that of LDA and RF unless the data variability is too high and/or effect sizes are too small. RF was found to outperform only kNN in some instances where the data are more variable and have smaller effect sizes, in which cases it also provide more stable error estimates than kNN and LDA. Applications to a number of real datasets supported the findings from the simulation study

Modeling concept drift: A probabilistic graphical model based approach

An often used approach for detecting and adapting to concept drift when doing classi cation is to treat the data as i.i.d. and use changes in classi cation accuracy as an indication of concept drift. In this paper, we take a different perspective and propose a framework, based on probabilistic graphical models, that explicitly represents concept drift using latent variables. To ensure effcient inference and learning, we resort to a variational Bayes inference scheme. As a proof of concept, we demonstrate and analyze the proposed framework using synthetic data sets as well as a real fi nancial data set from a Spanish bank

AI Researchers, Video Games Are Your Friends!

If you are an artificial intelligence researcher, you should look to video games as ideal testbeds for the work you do. If you are a video game developer, you should look to AI for the technology that makes completely new types of games possible. This chapter lays out the case for both of these propositions. It asks the question "what can video games do for AI", and discusses how in particular general video game playing is the ideal testbed for artificial general intelligence research. It then asks the question "what can AI do for video games", and lays out a vision for what video games might look like if we had significantly more advanced AI at our disposal. The chapter is based on my keynote at IJCCI 2015, and is written in an attempt to be accessible to a broad audience.Comment: in Studies in Computational Intelligence Studies in Computational Intelligence, Volume 669 2017. Springe

Relative blocking in posets

Poset-theoretic generalizations of set-theoretic committee constructions are presented. The structure of the corresponding subposets is described. Sequences of irreducible fractions associated to the principal order ideals of finite bounded posets are considered and those related to the Boolean lattices are explored; it is shown that such sequences inherit all the familiar properties of the Farey sequences.Comment: 29 pages. Corrected version of original publication which is available at http://www.springerlink.com, see Corrigendu

Improved CTA Coronary Segmentation with a Volume-Specific Intensity Threshold

State-of-the-art CTA imaging equipment has increased increased clinician's ability to make non-invasive diagnoses of coronary heart disease; however, an effective interpretation of the cardiac CTA becomes cumbersome due to large amount of imaged data. Intensity based background suppression is often used to enhance the coronary vasculature but setting a fixed threshold to discriminate coronaries from fatty muscles could be misleading due to non-homogeneous response of contrast medium in CTA volumes. In this work, we propose a volumespecific model of the contrast medium in the coronary segmentation process to improve the segmentation accuracy. The influence of the contrast medium in a CTA volume was modelled by approximating the intensity histogram of the descending aorta with Gaussian approximation. It should be noted that a significant variation in Gaussian mean for 12 CTA volumes validates the need of volume-wise exclusive intensity threshold for accurate coronary segmentation. Moreover, the effectiveness of the adaptive intensity threshold is illustrated with the help of qualitative and quantitative results

Radiographic Image Enhancement by Wiener Decorrelation

The primary focus of the application of image processing to radiography is the problem of segmentation. The general segmentation problem has been attacked on a broad front [1, 2], and thresholding, in particular, is a popular method [1, 3-6]. Unfortunately, geometric unsharpness destroys the crisp edges needed for unambiguous decisions, and this difficulty can be considered a problem in filtering in which the object is to devise a high-pass (sharpening) filter. This approach has been studied for more than 20 years [7-13]